what you are making here could be called a sparse trie . optional proofTrie argument so you can build onto an existing sparse trie is you want.

its kindof like the verification is done in this step because when you traverse the tree you already know each key is the hash of the node it's storing

This is the line that makes my tests pass, but I added the line to anywhere _lookupNode is being called (except in checkRoot which I believe is expected to just return false in that case).

So probably it could use some more tests that make sure it's properly propagating the other errors (but current tests continue to pass).

note The behavior here has changed slightly, but reading the objective of the test I think it's ok

You're including embedded nodes (specifically those with length < 32) in the proof in comparison with the previous prove method. Is that necessary for the non-existence proofs?

Shouldn't root of proofTrie be checked against rootHash here?

Actually yeah that's true. It's hard to say who's job this is. But if it's going to take the argument rootHash it should verify it. somehting like this will do it

  static verifyProof (rootHash, key, proofNodes, cb) {
    let proofTrie = new Trie(null, rootHash)
    Trie.fromProof(proofNodes, (error, proofTrie) => {
      if (error) cb(new Error('Invalid proof nodes given'), null)
      proofTrie.get(key, (e, r) => {
        return cb(e, r)
      })
    }, proofTrie)

db.get doesn't return an error if value is not found (just null for value). I wouldn't remove this if there's no special reason for this, as I can imagine in future db.get could potentially return errors for other reasons (e.g. validation of input).

db.get doesn't return an error if value is not found
There is a special reason for it. The way in which this tree is traversed and the property that is is a merkle tree, the child node must always exist in a valid tree.

I can imagine in future db.get could potentially return errors for other reasons
Im still returning that error below. Im just not blowing up as before (throwing an error during async function was just unhelpful blow up)

Hm, I wish you had simply added new test cases instead of modifying existing ones. Are these 3 cases not valid anymore? Going through them I have a hard time distinguishing between different result of verifyProof:

• If proof is for a key that doesn't exist and verifyProof checks the same key, val and err are null
• If proof is for a key and verifyProof checks a totally random key (that wasn't in proof), there'll be error
• In the first test case, proof is for key2bb, then it tries to verifyProof for key2, and it again returns null for val and err. I.e. the same proof is also a valid proof of absence for key2, which is certainly interesting, although maybe unexpected?

I'm not sure if users would need to distinguish between all the various states? One benefit of the previous approach in my eyes is that only the value for which the proof was generated would be verified.

If proof is for a key that doesn't exist and verifyProof checks the same key, val and err are null

the proof should prove the key is associated with null. It returns the correct value -> null, and it does not raise an error, because the verify function, having been called on that key, returned the sufficient nodes to prove this.

If proof is for a key and verifyProof checks a totally random key (that wasn't in proof), there'll be error

since this proof does not contain sufficient nodes to prove the random key, verifyProof(randomKey) will return an error. It does not matter if randomKey exists or not. It matter that the proof contains the correct nodes to verify such.

In the first test case, proof is for key2bb, then it tries to verifyProof for key2, and it again returns null for val and err. I.e. the same proof is also a valid proof of absence for key2, which is certainly interesting, although maybe unexpected?

Yes, this was an interesting edge-case that I wanted to test. I did not properly document it. In this case, the proof happens to contain enough nodes to prove the value at key2 because traversing into key22 would touch all the same nodes as traversing into key2 (and an extra). So the proof from key22 will be valid to prove the value of any shorter key.

One benefit of the previous approach in my eyes is that only the value for which the proof was generated would be verified

I dont think thats a benefit except maybe for testing. It also has a major drawback in that it does not allow proofs to be efficiently combined. This approach allows the "proofTree" to be drop-in replacement for a "regular tree" (it can even do put). A light client could in the future process state transitions using existing EVM software.